Search Results: "marco"

7 February 2014

Jo Shields: Dear Fake Debian Developers, shoo.

Another post about the Valve/Collabora free games thing. This time, the bad bit people trying to scam free games from us. Before I start, I want to make one thing clear there are people who have requested keys who don t meet the criteria, but are honest and legitimate in their requests. This blogspam is not about them at all. If you re in that category, you re not being complained about. So. Some numbers. At time of writing, I ve assigned keys to 279 Debian Developers or Debian Maintainers almost 25% of the total eligible pool of about 1200. I ve denied 22 requests. Of these 10 were polite requests from people who didn t meet the conditions stated (e.g. Ubuntu developers rather than Debian). These folks weren t at all a problem for us, and I explained politely that they didn t meet the terms we had agreed at the time with Valve. No problem at all with those folks. Next, we have the chancers, 7 of them, who pretended to be eligible when they clearly weren t. For example, two people sent me signed requests pointing to their entry on the Debian New Maintainers page when challenged over the key not being in the keyring. The NM page showed that they had registered as non-uploading Debian Contributors a couple of hours previously. A few just claimed I am a DD, here is my signature when they weren t DDs at all. Those requests were also binned.
Papers, Please screenshot - denied entry application

DENIED

And then we move onto the final category. These people tried identity theft, but did a terrible job of it. There were 5 people in this category:
From: Xxxxxxxx Xxxxxx <xxxxxxxx.xxxxxx@ieee.org>
Subject: free subscription to Debian Developer
8217 A205 5E57 043B 2883 054E 7F55 BB12 A40F 862E
This is not a signature, it s a fingerprint. Amusingly, it s not the fingerprint for the person who sent my mail, but that of Neil McGovern a co-worker at Collabora. Neil assured me he knew how to sign a mail properly, so I shitcanned that entry.
From: "Xxxxx, Xxxxxxxxx" <x.xxxxx@bbw-bremen.de>
Subject: Incoming!
Hey dude,
I want to have the redemption code you are offering for the Valve Games
mQGiBEVhrscRBAD4M5+qxhZUD67PIz0JeoJ0vB0hsLE6QPV144PLjLZOzHbl4H3N
...snip...
Lz8An1TEmmq7fltTpQ+Y1oWhnE8WhVeQAKCzh3MBoNd4AIGHcVDzv0N0k+bKZQ=3D=3D
=3Du/4R
Wat? Learn to GPG!
From: Xxxxxx-Xxxx Le Xxxxxxx Xxxx <xx.xxxxxxxxx@gmail.com>
Subject: pass steam
Hey me voila
Merci beaucoup
valve
2069 1DFC C2C9 8C47 9529 84EE 0001 8C22 381A 7594
Like the first one, a fingerprint. This one is for S bastien Villemot. Don t scammers know how to GPG sign?
From: "Xxxxxxxxx Xxxxxxx" <xxxxxxxx@web.de>
Subject: thanks /DD/Steam gifts us finally something back
0x6864730DF095E5E4
Yet again, a fingerprint. This one is for Marco Nenciarini. I found this request particularly offensive due to the subject line the haughty tone from an identity thief struck me as astonishingly impertinent. Still, when will scammers learn to GPG?
From: Sven Hoexter <svenhoexter@gmail.com>
Subject: Valve produced games
I'm would like to get the valve produced games
My keyring: 0xA6DC24D9DA2493D1 Sven Hoexter <hoexter> sig:6
Easily the best scam effort, since this is the only one which both a) registered an email address under the name of a DD, and b) used a fingerprint which actually corresponds to that human. Sadly for the scammer, I m a suspicious kind of person, so my instinct was to verify the claim via IRC.
31-01-2014 16:52:48 > directhex: Hoaxter, have you started using gmail without updating your GPG key? (note: significantly more likely is someone trying to steal your identity a little to steal valve keys from collabora)
31-01-2014 16:54:51 < Hoaxter!~sh@duckpond6.stormbind.net: directhex: I do not use any Google services and did not change my key
So yeah. Nice try, scammer. I m not listing, in all of this, the mails which Neil received from people who didn t actually read his mail to d-d-a. I m also not listing a story which I ve only heard second ha actually no, this one is too good not to share. Someone went onto db.debian.org, did a search for every DD in France, and emailed every Jabber JID (since they look like email addresses) asking them to forward unwanted keys. All in all, the number of evildoers is quite low, relative to the number of legitimate claims 12 baddies to 279 legitimate keys issued. But still, this is why the whole key issuing thing has been taking me so long and why I have the convoluted signature-based validation system in place. Enjoy your keys, all 279 of you (or more by the time some of you read this). The offer has no explicit expiry on it Valve will keep issuing keys as long as there is reason to, and Collabora will continue to administer their allocation as long as they remain our clients. It s a joint gift to the community thousands of dollars worth of games from Valve, and a significant amount of my time to administer them from Collabora.

30 January 2014

Marco d'Itri: On people totally opposed to systemd

Do you remember the very vocal people who, a decade ago, would endlessly argue that udev was broken and that they would never use it? Percentage over time of systems on which udev is installed Sometimes you can either embrace change or be dragged along by it. We are beyond the inflection point, and the systemd haters should choose their place.

28 January 2014

Andrea Colangelo: The (brand new) Debian 3D-Printing Team is hiring!

The 3D-printing technology has gained quite a big momentum in recent times. Its use is spreading widely even among hobbyists and many sources report daily several (sometimes weird) examples of its usage. The thing I like most, as a Free Software activist, is that several 3D-printing-related free and open source software are already available and quite good, and some printers are even released as free hardware. Unfortunately, Debian is quite lagged behind with regard to the availability of such software in our archive, especially compared to other distributions who already ship much of it. Hopefully this situation will change drastically thanks to the brand new Debian 3D-Printing team, whose purpose is to make Debian a rocking platform to do some serious 3D-printing. The team is bootstrapping right now, and we are looking for volunteers who are willing to help us. Packagers are greatly welcomed, but also triagers, users and passionate people with some experience in the field are extremely valuable. People willing to join the team can take a look at our (in-progress) wiki page and apply on the team page on Alioth. As with any good packaging team, a mailing list is available to share ideas, feel free to subscribe it, and feel free to join #debian-3dprinting on OFTC if you want to hang out with other team members.

21 December 2013

Russell Coker: Links December 2013

Andres Lozano gave an interesting TED talk about the use of electrodes inside the brain (deep brain stimulation) to treat Alzheimers disease, Parkinson s disease, and depression [1]. Daniel Pocock wrote an interesting post commenting on some bad political decisions being made in Australia titled Evacuating Australia [2]. You can read that as a suggestion to leave Australia or to try and make Australia better. Marco Tempest gave an interesting TED talk about Nikola Tesla [3]. The presentation method is one that I ve never seen before so I recommend watching the talk even if you already know all about Tesla. Charmian Gooch gave an interesting TED talk about global corruption [4]. I think we need people to send the information on shell company ownership to organisations like Wikileaks. The punishment for leaking such information would be a lot less than Chelsea Manning is getting and the chance of getting caught is also low. Rich Mogul wrote an interesting and insightful article for Macworld about the Apple approach to security problems [5]. To avoid the problem of users disabling security features they work to make the secure way of doing things EASIER for the user. That won t work with all security problems but it s something we need to think about when working on computer security. Ray Raphael gave an interesting TED talk about the parts of the US revolution that don t appear in history books [6]. He warns the listener to beware of the narrative forms, but another way to interpret his talk is that you should present your version of history in the narrative form that is best accepted. That lesson is well known and it s easy to see history being deliberately distorted in most media outlets. Will Wright gave an interesting TED talk about how he designed the game Spore and his ideas about games in general [7]. Spore is a really good game. Chris Lintott gave an interesting TED talk about crowd-sourced astronomy titled How to Discover a Planet from Your Sofa [8]. He referenced the Zooniverse.org site which lists many crowd-sourced science projects [9]. Jake Socha gave an interesting TED talk about flying snakes, you have to see this to believe it [10]. Nikita Bier gave an interesting TED talk about his webapp to analyse economic policies [11]. Apparently 60% of people were going to vote in their best economic interest before seeing his site and 66% would do so afterwards that could change an election result. Anya Kamenetz wrote an interesting article for Salon about The Iliad Project which aims to use Indigogo to help identify new anti-biotics [12]. The current ways of discovering anti-biotics aren t working, lets hope this one does. Peter Finocchiaro wrote an interesting Salon article about how right-wing politicians in the US were opposed to Nelson Mandela [13] racism meets anti-communism. Katie McDonough wrote an interesting Salon article about Rick Santorum and Bill O Reilly comparing Obamacare to apartheid while supposedly honoring Nelson Mandela [14], Katie also notes that Nelson enshrined universal healthcare in the South African constitution something all countries should do. Mary Elizabeth Williams wrote a Salon article about Susan Boyle s announcement about being diagnosed with Asperger Syndrome [15]. Not a surprise though, some people can be diagnosed with Autism by merely watching them on TV. Amelia Hill wrote an article for The Guardian/The Observer about the educational results of Home Schooling [16]. Apparently Home-Schooled kids learn significantly more and home-educated children of working-class parents achieved considerably higher marks in tests than the children of professional, middle-class parents and that gender differences in exam results disappear among home-taught children . Wow, Home Schooling beats gender and class problems! I m sure it s even better for GLBT kids too. Robert Reich wrote an interesting Salon article about the way rich people in the US give tax-deductable (taxpayer supported) donations to charities that benefit themselves [17]. Dan Savage wrote a very funny review of Sarah Palin s latest Christmas book, one classic quote is why should I have to read the whole thing? Lord knows Sarah Palin didn t write the whole thing [18]. He makes a good point that we should use the term happy holidays instead of happy Christmas just to show that we aren t assholes.

1 December 2013

Marco d'Itri: Easily installing Debian on a Cubieboard

I recently bought a Cubieboard to replace my old Sheevaplug which has finally blown a power supply capacitor (this appears to be a common defect of Sheevaplugs), so I am publishing these instructions which show how to install Debian on sunxi systems (i.e. based on the Allwinner A10 SoC or one of its newer versions) with no need for cross compilers, emulators or ugly FAT partitions. This should work on any sunxi system as long as U-Boot is at least version 2012.10. The first step is to erase the beginning of SD card to remove anything in the unpartitioned space which may confuse U-Boot, partition and format it as desired. The first partition must begin at 1MB (1024*1024/512=2048 sectors) because the leading unpartitioned space is used by the boot loaders.
dd if=/dev/zero of=/dev/mmcblk0 bs=1M count=1
parted /dev/mmcblk0
  mklabel msdos
  mkpart primary ext4 2048s 15G
  unit s
  print
  mkpart primary linux-swap ... -1
mkfs.ext4 -L root /dev/mmcblk0p1
mkswap --label swap /dev/mmcblk0p2
Download the boot loaders and an initial kernel and install them:
tar xf cubieboard_hwpack.tar.xz
dd if=bootloader/sunxi-spl.bin of=/dev/mmcblk0 bs=1024 seek=8
dd if=bootloader/u-boot.bin of=/dev/mmcblk0 bs=1024 seek=32
mount /dev/mmcblk0p1 /mnt
mkdir /mnt/boot/
cp kernel/script.bin kernel/uImage /mnt/boot/
script.bin is Allwinner's proprietary equivalent of the device tree: it will be needed until sunxi support will be fully merged in mainline kernels. U-Boot needs to be configured to load the kernel from the ext4 file system (join the lines at \\, this is not a supported syntax!):
cat << END > /mnt/boot/uEnv.txt
# kernel=uImage
root=/dev/mmcblk0p1 rootwait
boot_mmc=ext4load mmc 0:1 0x43000000 boot/script.bin && ext4load mmc 0:1 0x48000000 boot/$ kernel  \\
  && watchdog 0 && bootm 0x48000000
END
Now the system is bootable: add your own root file system or build one with debootstrap. My old Sheevaplug tutorial shows how to do this without a working ARM system or emulator (beware: the other parts are quite obsolete and should not be trusted blindly). If you have an old armel install around it will work as well, and you can easily cross-grade it to armhf as long as it is up to date to at least wheezy (the newer, the better). You can also just use busybox for a quick test:
mkdir /mnt/bin/
dpkg-deb -x .../busybox-static_1.21.0-1_armhf.deb .
cp bin/busybox /mnt/bin/
ln -s busybox /mnt/bin/sh
After booting the busybox root file system you can run busybox --install /bin/ to install links for all the supported commands. Until Debian kernels will support sunxi (do not hold your breath: there are still many parts which are not yet in mainline) I recommend to install one of Roman's kernels:
dpkg -i linux-image-3.4.67-r0-s-rm2+_3.4.67-r0-s-rm2+-10.00.Custom_armhf.deb
mkimage -A arm -O linux -T kernel -C none -a 40008000 -e 40008000 \
  -n uImage -d /boot/vmlinuz-3.4.67-r0-s-rm2+ /boot/uImage-3.4.67-r0-s-rm2+
It is not needed with these kernels for most setups, but an initramfs can be created with:
update-initramfs -c -k 3.4.67-r0-s-rm2+
mkimage -A arm -T ramdisk -C none -n uInitrd \
  -d /boot/initrd.img-3.4.67-r0-s-rm2+ /boot/uInitrd-3.4.67-r0-s-rm2+
/boot/uEnv.txt will have to be updated to load the initramfs. Since the Cubieboard lacks a factory-burned MAC address you should either configure one in script.bin or (much easier) add it to /etc/network/interfaces:
iface eth0 inet dhcp
        hwaddress ether xx:xx:xx:xx:xx:xx
To learn more about the Allwinner SoCs boot process you can consult 1 and 2.

2 November 2013

Marco d'Itri: New PGP key

Since my current PGP key is a 1024 bits DSA key generated in 1998, I decided that it is time to replace it with a stronger one: there are legitimate concerns that breaking 1024 bits DSA is well within the reach of major governments. I have been holding out for the last year waiting for GnuPG 2.1, which will support elliptic curves cryptography, but I recently concluded that adopting ECC now would not be a good idea: Red Hat still does not fully support it due to unspecified patent concerns and there is no consensus in the cryptanalists community about the continued strength of (some?) ECC algorithms. So I created three fancy keys: a 4096 bits main key for offline storage, which hopefully will be strong enough for a long time, and two 3072 bits subkeys for everyday use. I have published a formal key transition statement and I will appreciate if people who have signed my old key will also sign the new one. What follows are the instructions that I used to generate these PGP keys. They follow the current best practices and only reference modern software. While the GnuPG defaults are usually appropriate, I think that it is a good idea to use a stronger hash for the key signatures of very long-lived keys. I could not find a simple way to "upgrade" the algorithm of key self signatures.
echo 'cert-digest-algo SHA256' >> ~/.gnupg/gpg.conf
First, generate a RSA/4096 sign only key, which will be your master key and may be stored offline. Then add to it two RSA/3072 subkeys (one sign only and one encrypt only):
# generate a RSA/4096 sign only key
gpg2 --gen-key
# add two RSA/3072 subkeys (sign only and encrypt only)
gpg2 --edit-key 8DC968B0
Since GnuPG lacks a command to remove the master secret key while keeping its secret subkeys, you need to delete the complete secret keys and then re-import only the subkeys:
gpg2 --export-secret-keys 8DC968B0 > backup.secret
gpg2 --export-secret-subkeys 8DC968B0 > backup.subkeys
gpg2 --delete-secret-key 8DC968B0
gpg2 --import backup.subkeys
Then you can import again the complete keys in a different secret keyring, which can be stored offline:
mkdir ~/.gnupg/master/
gpg2 --no-default-keyring \
  --keyring ~/.gnupg/pubring.gpg \
  --secret-keyring ~/.gnupg/master/secring.gpg \
  --import backup.secret
Now you can move ~/.gnupg/master/ to a USB stick. You are supposed to protect the master secret key with a strong passphrase, so there is no point in using block level encryption on the removable media. Since you are only using the master key to sign other keys, it only needs to be configured as the second keyring in ~/.caffrc:
$CONFIG 'secret-keyring'  = $ENV HOME  . '/.gnupg/master/secring.gpg';
It is also a good idea to have an hard copy backup of your keys, since the lifetime of USB sticks should not be trusted too much:
paperkey -v --output printable.txt --secret-key backup.secret
a2ps -2 --no-header -o printable.ps printable.txt
Some references that I used:

8 October 2013

Thomas Goirand: OpenStack 2013.2~rc1, aka Havana, fully available in Debian Experimental

Announcement After a very long work, over the course of 4 months, I have finished packaging the first RC1 of OpenStack. This comes on time, just 9 days before the official Havana release. Please do try this RC1 before the official 2013.2, code name Havana, is released, and hopefully uploaded to Debian. All of the packages are available from Debian Experimental, keeping Grizzly in Sid. However, there is also some private repositories that I maintain, holding Wheezy backports: deb http://havana.pkgs.enovance.com/debian havana main deb http://archive.gplhost.com/debian havana-backports main The first repository holds the packages maintained within the Alioth group. These are built directly from my Jenkins machine, on each git push. The 2nd repository is holding backports from Sid to Wheezy for the packages that I don t actively maintain (though a lot of them are in the Python module team, in which I do a lot of packaging and updates as well). A few numbers A few numbers about this now. I had to work on 145 source packages: at least backport them to Wheezy, and push them in the GPLHost archive repository above. This generates 360 binary packages. Out of these, I maintain 77 source packages within the Alioth OpenStack group, generating 209 .deb files. That s a lot of stuff to deal with (and I feel sometimes a bit dizzy about it). While OpenStack is a big jigsaw puzzle to solve for the users, it is even more for someone who has to deal with all the (sometimes buried in the the code) Python dependencies. I hope others will come and join me in this packaging effort, since over the time, there s more and more work to be done, as the project grows. Note that most of the work is unfortunately done on packaging (and updating) the Python dependencies, working on the packages themselves is done last, at the end of the cycle. Other things not packaged (yet) Before the release (and the forthcoming Hongkong summit on the 5th of November), I hope to be able to finish packaging TripleO. TripleO is in fact OpenStack on OpenStack, which works with nova-baremetal. I have no idea how to test or install this, though it sounds like a lot of fun. There are 6 source packages that need to be done. Also, pending on the FTP masters NEW queue, is Trove: Database as a Service. I hope this one can get through soon. There is, also, Marconi, which is an incubated project for a new message queuing service, which probably will replace RabbitMQ (I m not sure yet what it does, and I will be happy to hear about it at the summit). Lastly, there s Ironic, which will at some point, replace nova-baremetal. That is, it does cloud computing over bare metal, without virtualization. All of these new projects are still in an incubation stage, and are not part of the official release yet. Though, I have learned over the course of this past year, that with OpenStack, it s never early enough to start the packaging work. Thanks to my sponsor! Please note that all of this wouldn t be possible without eNovance sponsoring my packaging work. A big up to all of them for supporting and loving Debian! You guys rox. Also a special thanks to Mehdi / Sileth, for his work testing everything with the Tempest functional tests and the CI platform.

22 July 2013

Daniel Pocock: Winning at any cost

It's not every day that a student messing around with keystroke loggers comes to fame through slashdot. Nonetheless, systematically rigging an election and getting sentenced to 12 months in a dorm with bars has helped raise 22 year old Matthew Weaver's profile well above that of the average script kiddie. Now let's stop and reflect on poor Weaver's future. You may be thinking that with an exchange program like this on his academic record he won't be so popular with employers. Given that he was busted by campus security rather than the FBI he won't even attract the interest of those companies who hire ex-hackers. So where could he go? How is it done in Australia? Not too long ago, when I was a student myself, one of our prominent universities was subjected to a very similar scam. Four members of the Tin Tin for NUS ticket at La Trobe University were implicated in stuffing the ballot the old fashioned way. The incidents even share the characteristics of chronic stupidity: just as Weaver had been caught voting for himself 259 times from the same IP address in a campus computer lab, team Tin Tin had tried to hand their bag of manipulated postal votes directly to the deputy returning officer rather than discretely posting them through the internal mail. According to an official report by the Deputy Returning Officer, Karsten Haley, all four candidates were charged with Dishonest Conduct and Interfering with Ballot Papers. Unfortunately, the report notes that
La Trobe University SRC Electoral Regulations do not empower the Returning Officer or Deputy to enforce charges or disciplinary procedures and the charges were never faced by the accused.
Given the seriousness of the matter, Haley did not give up his attempts to hold them to account. He escalated it to the Dean of the college and then to the University Secretary. He reports that "their disinterest was extraordinary" and that nobody would involve the police. Young Labor suspended Just over a year later, in 1997, the ALP's youth division for the state of Victoria, Young Labor, was suspended after attempts to rig the ballot to elect the Young Labor leadership team. The guilty parties were never publicly named. Nobody was formally suspended or expelled and this simply left them with more time on their hands to invest their energy in other elections. The suspension of Victorian Young Labor remained in effect for a number of years. The specific allegations about the Young Labor ballot suggest that those people particularly keen to win had printed fake student cards and given them to stooges who would impersonate other Young Labor members who had not attended to vote in person. Where are they now? It is no co-incidence that these students were (and still are) members of Labor Unity, a powerful faction within Australia's ruling Labor Party, the ALP. Most political organisations would presumably express concern about these allegations. The ALP does things differently. One of the students who withdrew his nomination in La Trobe, Mr Larocca, subsequently became Mayor in the City of Moreland, one of the ALP's strongholds. Even more remarkably from an outsider's viewpoint, another of these figures, Stephen Donnelly, is currently employed as the Assistant State Secretary of the ALP in Victoria. Communications like this newsletter reveal that he is one of the key figures in the party's pre-selection process. He has recently been appointed to direct the ALP's 2013 federal election campaign for the state of Victoria. Another co-incidence On the same weekend that Weaver was in the news for his antics, Donnelly's latest employer, the ALP's Victorian branch, was conducting pre-selection ballots to choose candidates for the upcoming federal election. So it's no surprise that Monday's newspaper headlines report fresh allegations of voting irregularities. Sadly, I've seen some of Labor Unity's bad behavior first hand. About 10 years ago I was living in South Melbourne, which is in the federal electoral district of Melbourne Ports. A young female friend of mine, a member of the local Elwood branch of the ALP, had spent election day handing out brochures for an ALP candidate in a marginal seat rather than assisting the controversial local ALP candidate, Michael Danby. A few days later I was witness to an incident where Danby aggressively confronted this young woman and demanded to know why he hadn't seen her handing out his own leaflets on polling day. He stood within centimeters of her and was literally looking down on her as he demanded some kind of apology to sooth his bruised ego. She looked terrified and barely responded. Within moments one of his handlers approached and physically moved Danby away from this young woman, I dare to think where things would have gone otherwise. Eye for talent Remarkably, at the same time, the infamous Stephen Donnelly had started shadowing Danby in his movements about the district. Fresh out of university, his talents had been recognised by Danby and he was employed in Danby's office, enabling him to continue honing his skills on a full-time basis with a tax-payer funded salary. What a remarkable contrast to the story of Weaver. Can anybody imagine a US congressman collecting Weaver from the prison gates and deploying him to an office on Capitol Hill? The biggest bankruptcy in student history Around the same time, Donnelly's Student Unity, the student arm of Labor Unity were successful in taking over the student union of my own campus, the University of Melbourne. Not long after I graduated I heard that they had been accused of skimming off $1 million from catering providers and a high-risk $46 million property transaction that put the organisation into liquidation. Unlike Mr Weaver, who's scheme at Cal State barely got off the ground, none of those involved in the Melbourne University incident has faced criminal proceedings. One ALP figure, Andrew Landeryou, spent several months in Costa Rica while wanted for questioning in the Supreme Court. His wife has just been endorsed for a seat in the Senate with support from various Labor Unity figures including Danby. The Gillard questions In 1996, around the same time that Donnelly & Co. were romping around student unions learning the tricks of the political trade, a lawyer quietly departed from the firm Slater and Gorden after an internal investigation into a property transaction linked to a union slush fund. Like Donnelly, this lawyer's next move was to take employment in the office of a Labor Party MP. More recently she was backed by Labor Unity to become Prime Minister. The union slush fund remains under investigation, frustrated by the disappearance of documents. The $60 million heist Recently I blogged about Gillard and Abbott, leaders of the two main political parties in Australia, agreeing to take $60 million of taxpayer money to fund their parties' campaigns in the upcoming federal election, giving themselves an obscenely unfair unadvantage over all other contestants. Where would that money end up? In the case of the ALP, does it appear likely that figures like the Victorian ALP's federal campaign director, Mr Donnelly, would be involved in the expenditure? National shame With this background, it becomes easier to understand the quality (or lack of it) in Australia's national leadership. When you consider that the generation responsible for the La Trobe incident, the Young Labor suspension and the MUSU bankruptcy are now growing into positions of greater responsibility in the ALP it leaves me feeling the quality of leadership is only going to get a lot worse before it starts getting better. For example, the recent incident where coloured people were fed to the sharks has nothing to do with the worldwide refugee crisis and everything to do with maintaining the dumbed-down level of political discourse that Labor Unity thugs and their followers can cope with. Real issues like climate change and energy policy, for example, appear to be beyond the pay grade of Australia's political class Ranjini - coloured, indefinite detention It is startling that up to her own recent demise, Gillard herself had repeatedly begged the public to stop asking questions about her own past and remember that Labor politicians are innocent until proven guilty - yet she had a pregnant coloured woman thrown into a concentration camp on unfounded fears about "national security". No evidence has ever been presented that poor Ranjini committed a crime, but the houses bought with money from trade unions, transactions handled through Gillard's own office, seem to be as solid as bricks and mortar. If only poor Matthew Weaver had been an Australian, how much further would his star have risen? Update: please sign the petition at change.org asking La Trobe university to re-examine the report and refer it formally to the police. If you are concerned about the plight of poor Ranjini and other people subject to Australia's domestic rendition program, please take a moment to see Letters for Ranjini

7 July 2013

Raphael Geissert: Dealing with bashisms in proprietary software

Sometimes it happens that for one reason or another there's a need to use a proprietary application (read: can not be modified due to its licence) that contains bashisms. Since the application can not be modified and it might not be desirable to change the default /bin/sh, dealing with such applications can be a pain. Or not.

The switchsh program (available in Debian) by Marco d'Itri can be used to execute said application under a namespace where bash is bind-mounted on /bin/sh. The result:

$ sh --help
sh: Illegal option --
$ switchsh sh --help head -n1
GNU bash, version 4.1.5(1)-release-(i486-pc-linux-gnu)

Simple, yet handy.

7 June 2013

Marco d'Itri: Torre Telecom Italia, Rozzano

Today I was lucky enough to be able to visit the Telecom Italia telecommunications tower located in Rozzano, just south of Milano, and took some photos. This tower, with its 187 meters, is one of the tallest man-made structures in Italy. It was built by Telecom Italia in 1990 to create high capacity radio links to Genova and Torino and nowadays it contains radio transmitters for a TV station and many kinds of radio networks. It is an impressive monument to an age when telcos had no optical fibers, but plenty of money.

27 March 2013

Bits from Debian: Debian Project Leader elections 2013: interview with Lucas Nussbaum

We have asked Lucas Nussbaum, one of the three candidates for DPL elections 2013, to tell our readers about himself and his ideas for the Debian Project. You can also read the interviews to the other two candidates: Gergely Nagy and Moray Allan.
Please tell us a little about yourself. Hi! I'm a 31 years old french computer geek. In my day job, I'm an assistant professor (Ma tre de Conf rences) of Computer Science at Universit de Lorraine. What do you do in Debian and how did you started contributing? Like many, I started contributing to Debian by creating and maintaining packages for my own software, in the Ruby team. Then, I discovered that, even if it's not so obvious from the outside, there are a lot of areas in Debian that could use more contributors. So I just started to contribute to more and more things. There's a list of things I did in Debian in my platform. What I have been doing recently is: Why did you decide to run as DPL? Two main reasons: Three keywords to summarise your platform. (re-)make Debian the center of the Free Software ecosystem; foster innovation inside Debian; reduce barriers to contributions What are the biggest challenges that you envision for Debian in the future? I often have the impression that the project is losing momentum, positive energy, and slowing down. It feels like we are living on the benefits of the past. A lot of very cool things happen in the Debian ecosystem, but very often outside the Debian project (in derivative distributions). Debian should aim at reinforcing its position in the center of the Free Software ecosystem: it should be the main active intermediary between upstream projects and final users. To achieve that, we need to reinforce the visibility and the impact of Debian. This is extremely important because the values we fight for as a project are often neglected by our derivatives. What are, in your opinion, the areas of the project more in need of technical and/or social improvements? Fostering innovation inside Debian: we should be more welcoming towards innovation and experiments inside the project. Often, we merely tolerate them, and bureaucracy makes them hard and slow to conduct. As a result, people tends to innovate outside the Debian project. Making it easier to contribute to Debian: we compete with more and more projects to attract contributors. While we are already quite good at welcoming new contributors with good documentation and mentoring (much better than people usually think), there's still a lot of room for improvement. Why should people vote for you? A great thing in Debian's voting system is that you don't vote "for" or "against" a specific candidate. Instead, due to our use of the Condorcet method, you rank candidates (and also indicate those who you consider suitable for the role by ranking a virtual "None of the above" candidate). Why am I a good candidate? My previous contributions to Debian show that I have a pretty good understanding of the inner workings of the project, and that I have a track record of managing projects successfully inside Debian. I think that those are two required qualities for a DPL. Name three tools you couldn't stay without. vim, mutt, ssh. What keep you motivated to work in Debian? Debian is a fantastic project from a technical point of view (focus on technical excellence, lots of interesting challenges), but also from a social point of view: the Debian community is a great community where I have lots of good friends. Also, what's great when you contribute to Debian is that your work has a real impact, and that you see people using stuff you worked on everywhere. Are there any other fields where you call yourself a geek, besides computers? I'm not sure this really qualifies as "besides computers", but I've gotten very interested in the OpenStreetMap project lately. I very much enjoy exploring unmapped areas on a mountain bike. It feels like being Christopher Columbus or Marco Polo, but 20 minutes from home. ;) The OpenStreetMap and Debian projects also share many values, such as a great attention to quality and details.

9 November 2012

Gunnar Wolf: Road trip to ECSL 2012 in Guatemala

Encuentro Centroamericano de Software Libre! Guatemala! During a national (for us) holiday, so it's easy to go without missing too much work time! How would I miss the opportunity? Several years ago, I started playing with the idea of having a road trip Probably this was first prompted with the UK crew and the Three Intrepid Motorcycle Riders arriving by land to DebConf 9 I don't know. Fact is, I wanted to go to DebConf10 in New York by land, as well as to DebConf12 in Nicaragua. Mostly due to a lack of time, I didn't Although we did start making some longish trips. Of course, my desire to show Regina what Mexico is like also helped! So, up until a week ago, our (according to my standards) long distance driving experience included:
  • M xico Guanajuato Puerto Vallarta Guanajuato M xico, in early November 2011, for Festival de Software Libre and with Regina and our Blender friends Octavio and Claudia. Totalling almost 1900Km, mostly consisting of wide, toll highway.
  • M xico Xilitla San Luis Potos M xico, in April 2012, just for fun and for a nice short vacation, alone with Regina. Totalling almost 1200Km, but through Sierra Gorda de Quer taro, a very tough stretch of about 250Km which we did at about 50Km/h on average. Beautiful route for sure! We didn't originally intend to go through San Luis Potos , and it does not appear to make much sense, as it adds ~350Km to the total, but it was even quicker than going back by the same route and according to those who now, even faster than our planned route, via Tamazunchale and Ixmiquilpan!
  • M xico San Luis Potos Zacatecas Aguascalientes Guanajuato M xico, in May 2012, for Congreso Internacional de Software Libre, again with Octavio and Claudia. Totalling 1250Km, and following very good roads, although most of them were toll-free.
But there is always a certain halo over crossing a border, maybe more so in countries as large as Mexico. We convinced Pooka and Moni, and granted, with some aprehension, as we knew of some important security risks in the more rural areas we wanted to go through we decided to go to Guatemala. And, although we wanted to go with a bit more time, Real Life took its toll: We could not take more time than the intersection of what our respective jobs offered. So, here goes a short(?) recap of our six day long, 3200Km trip. Of course, we have a map detailing this. Mexico Veracruz I came to my office early on Wednesday (31-oct), and left with Regina around 10AM towards Veracruz. We agreed to meet there with Moni and Pooka, who would take the night bus, and continue together. Crossing Mexico City proved to be the longest obstacle We arrived to Veracruz already past 3PM, and spent a nice evening walking down the center and port of the city. Veracruz port can still be seen as part of central Mexico; I knew the road quite well. Veracruz San Andr s Tuxtla Catemaco San Cristobal de las Casas We met with our friends at the iconic Gran Caf de la Parroquia at 6:30AM. Had a nice breakfast with coffee, and by 7:30 we were heading south-west. The reason to have a road trip was to get to know the route, to enjoy the countryside So, given we "only" had to make 650Km this day, we took the non-toll road A narrow path stretching along the coastal plains of Veracruz, until Acayucan. Doing so, we also saved some money, as the equivalent toll road is around MX$300 (~US$25)! Veracruz is a hot state. We ended up all sweaty and tired by 19:00, when we reached San Cristobal. We had agreed not to drive at night, due to security issues, but fortunately there was quite a bit of traffic both ways between Tuxtla Guti rrez (Chiapas State capital, around 1hr from San Cristobal, where darkness got us) and our destination, so we carried on. Now, San Cristobal is a high city, almost as high as Mexico City (2100m), and being more humid, it was quite chilly. We went for a walk, and were convinced that at a later time, we had to stay for several days there. The city is beautiful, the region is breath-taking, there is a lot of great handicrafts as well, and it's overall very cheap. Really lovely place. San Cristobal de las Casas Cd. Cuauht moc La Mesilla Guatemala Once again, this day started early. We woke up ready to leave at 7AM, and not earlier because the hotel's parking didn't open earlier. After a very quick visit to San Cristobal downtown, to take some photos that were not right the night before, we took the road to Comit n, stopping just for some tamales de bola y chipil n for breakfast. Central Chiapas is almost continuously populated, differing from most of my experience in Mexico. It is all humid, and has some very beautiful landscapes. We passed Comit n, which is a much larger city than what we expected, went downhill after La Trinitaria, crossed a plain, and continued until hills started taking over again. We stopped in a very chaotic, dirty place: Just accross the border, where Ciudad Cuauht moc becomes La Mesilla. This border was basically what we expected: There is no half-official place to exchange money, so we had to buy quetzales from somebody who offered them on the street, at MX$2 per Q1 (where the real exchange should be around 1.50 to 1). While on the road, I was half-looking for exchange posts in Comit n and onwards, and found none (and being a festive day, they would probably be closed anyway). But we were expecting this, after all, and exchanged just the basic minimum: MX$600 (US$50, which by magic became Q300, US$40). The tramit consists of:
  • Spraying the car against diseases (which has a cost of Q18)
  • Each of us has to go through migration. Note, in case you cross this border: We didn't expressly cross Mexican migration, so officially there was no record of us going out. Be sure to go through migration to avoid problems at re-entry!
    Migration has no cost.
  • Customs. As we were entering by car, I had to purchase a permit for circulation. I don't remember the exact quote, but it was around Q150, and the permit is valid for 90 days.
  • That's it! Welcome to Guatemala!
La Mesilla is in Guatemala's Huehuetenango Department, and from all of the Departments we crossed until Guatemala city (Huehuetenango, Quetzaltenango, Totonicap n, Solol , Chimaltenango, Sacatep quez and Guatemala), this is the largest one. Huehuetenango is home to the Cuchumatanes mountain ridge. We found beautiful, really steep, really fertile mountains. It is plainly amazing: Mountains over 60 , and quite often full with agricultural use Even at their steepest points! The CA-1 highway is, in general, in very good shape. There are however many (many, many) speed bumps (or topes, in Mexican terminology. Or t mulos in Guatemalan), at least a couple at every village we crossed, not always painted. The road is narrow and quite winding; it follows river streams for most of the way. We feared it would be in much worse shape, from what we have heard, but during the whole way we found only three points where the road was unusable due to landslides and an alternative road was always in place when we needed it. After Totonicap n, the narrow road becomes a wide (four lane) highway. Don't let that fool you! It still goes through the center of every village along the road, so it's really not meant for speeding. Also, even though the pavement is in very good condition, it is really steep quite often. It is not the easiest road to drive, but it's (again) by far not as bad as we expected. We arrived to Guatemala City as dawn was falling, and got promptly lost. Guatemala has a very strange organization scheme: The city is divided in several zones, laid out in a swirl-like fashion. East-west roads are called Calle and North-south roads are called Avenida (except for zona 4, I think, where they are diagonal, and some are Rutas while the others are V as). I won't go into too much detail). Thing is, many people told us it's a foolproof design, and people from different countries understand the system perfectly. We didn't... At least not when we arrived. We got quite lost, and it took us around one hour to arrive to our hotel, at almost 19:00 Almost 12 hours since we left San Cristobal. Went for a quick dinner, and then waited for our friends to arrive after the first day of work of ECSL, which we missed completely. And, of course, we were quite tired, so we didn't stay up much longer. Antigua Guatemala On Saturday, ECSL's activities started after 14:00 so we almost-kidnapped Wences, the local organization lead, and took him to show us around Antigua Guatemala. Antigua was the capital of Guatemala until an earthquake destroyed it in the 1770s; the capital was moved to present-day Guatemala city, but Antigua was never completely abandoned. Today, it is a world heritage site, a beautiful city, where we could/should have stayed for several days. But we were there for the conference, so we were in Antigua just a couple of hours, and headed back to Guatemala. Word of caution: Going from Guatemala to Antigua, we went down via the steepest road I have ever driven. Again, a real four-lane highway... but quite scary! The main focus for this post is to give some roadtrip advice to potential readers... So, this time around, I won't give much detail regarding ECSL. It was quite interesting, we had some very good discussions... but it would take me too much space to talk about it! The road back: Guatemala Tec n Um n; Cd. Hidalgo Arriaga So, about the road back: Yes, we just spent three days getting to Guatemala City. We were there only for ~36 hours. And... We needed to be here by Tuesday morning no matter what. So, Sunday at noon we said goodbye to our good friends in ECSL and started the long way back. To get to know more of Guatemala, we went back by the CA-2 highway, which goes via the coastal plains Not close to the Pacific ocean, which we didn't get to see at all, but not through the mountains. To get to CA-2, we took CA-9 from Guatemala. If I am not mistaken, this is the only toll road in Guatemala (at least, the only we used, and we used some pretty good highways!) It is not expensive; I don't remember right now, but must have been around Q20 (US$3). Went South past Palin and until CA-2, just outside Escuintla city, and headed West. All of Escuintla and Suchitep quez it is again a four lane highway; somewhere in Retalhueu it becomes a two lane highway. We were strongly advised not to take this road at night because, as the population density is significantly lower than in CA-1, it can get lonely at times And there are several reports of robberies. We did feel the place much less populated, but saw nothing suspicious in any way. Something important: There are almost no speedbumps in CA-2! The terrain stayed quite flat and easy as we crossed Quetzaltenango, and only in San Marcos we found some interesting hills and a very strong rain that would intermitetntly accompany us for the rest of the ride. So, we finally arrived to the border city of Tec n Um n at around 16:30 Approximately four hours after leaving the capital. The Tec n Um n Cd. Hidalgo cities and border pass are completely different from the disorderly and dirty Cd. Cuauht moc La Mesilla ones. The city of Tec n Um n could be just a nice town anywhere in the country, it does not feel aggressive as most border cities I have seen in our continent. We stopped to eat at El pollo campero and headed to the border. In the Mexican side, we also saw a very well consolidated, big and ordered migration area. Migration officers were very kind and helpful As we left Cd. Cuauht moc, Regina didn't get a stamp of leaving Mexico, so technically she was ilegally out of the country (as she is not a national... They didn't care about the rest of us). The tramit to fix this was easy, simple, straightforward. We only paid for the fumigation again (MX$60, US$5), and were allowed to leave. Anyway, we crossed the border. There is a ~30Km narrow road between Cd. Hidalgo and Tapachula, but starting in Tapachula we went on Northwards via a very good, four lane and very straight highway. Even though we had agreed not to drive at night... Well, we were quite hurried and still too far from Mexico City, so we decided to push it for three more hours, following the coastline until the city of Arriaga, almost at the border between Chiapas and Oaxaca. Found a little hotel to sleep some hours and collapsed. Word of warning: This road (from Tapachula to Arriaga) is also known for its robberies. We saw only one suspicious thing: Two guys were pushing up their motorcycle, from which they had apparently fallen. We didn't stop, as they looked healthy and not much in need of help, but later on talked about this Even though this was at night, they were not moving as if they had just crashed; nothing was scratched, not the motorcycle and not their clothes. That might have been an attempt to mug us (or whoever stopped by). This highway is very lonely, and the two directions are separated by a wall of vegetation, so nobody would have probably seen us were we to stop for some minutes. Be aware if you use this road! The trip comes to an end: Arriaga Niltepec Istmo C rdoba M xico The next (last, finally!) day, we left at 6:30AM. After driving somewhat over one hour, we arrived to Niltepec, where a group of taxi drivers had the highway closed as a protest against their local government's tolerance of mototaxis. We evaluated going back to Arriaga and continue via the Tuxtla Guti rrez highway, but that would have been too long. We had a nice breakfast of tlayudas (which resulted in Pooka getting an alergic reaction shortly afterwards) and, talking with people here and there, were told about an alternative route by an agricultural road that surrounds the blockade. So, we took this road the best way we could, and after probably 1hr of driving at 20Km/h, finally came back to the main road. We planned on crossing the isthmus using the Acayucan-Juchit n road We were amazed at the La Ventosa ("the windy") area, where we crossed a huge eolic plant for electricity generation, so of course we got our good share of photos. From then onwards, not much more worth mention. Crossed the isthmus via a quite secondary road in not too good shape (although there is a lot of machinery, and the road will much likely improve in the next few months/years), then took the toll freeway along Veracruz until C rdoba. We stopped for a (delilcious and revigorizing!) cup of coffee in Hotel Zeballos, where Agust n de Iturbide signed with Viceroy Juan O'Donoj the treaties that granted Mexico the independence. Traveller, beware: When crossing between Puebla and Veracruz, there is a steep slope of almost 1000m where , you will almost always (except if it's close to noon) find very thick fog; taking the highway from C rdoba, this is in the region known as Cumbres de Maltrata. We had the usual fog, and just as we left it, a thin but constant rain that went on with us up until we got home. Crossed Puebla state with no further eventualities, and arrived to Pooka and Moni's house by 22:00. Less than one hour later, Regina and me arrived home as well. This was four days ago... and I have finally finished writing it all down ;-) Hope you find this useful, or if not, at least entertaining! If you read this post in my blog, you will find many pictures taken along the trip below (Well, if you are reading the right page, not in the general blog index...). If you are reading from a planet or other syndication service... Well, come to the blog! Dreamhost woes Oh, and... Yes, it sometimes happens: My blog is hosted at Dreamhost. This means that usually it works correctly... But sometimes, specially when many people request many nontrivial pages, it just gives an error. If you get an error, reload once or twice... Or until your patience manages ;-)

16 August 2012

Rapha&#235;l Hertzog: Happy Birthday Debian! And memories of an old-timer

For Debian s birthday, Francesca Ciceri of the Debian Publicity team suggested that developers blog about their first experiences with Debian . I found this a good idea so I m going to share my own early experience. It s quite different from what happens nowadays Before speaking of my early Debian experience, I have to set some context. In my youth, I have always been a Windows user and a fan of Bill Gates. That is until I got Internet at home at that point, I got involved in Usenet and made some friends there. One of those made me discover Perl and it has been somewhat of a revelation for me who had only been programming in Visual Basic, Delphi or ObjectPal. Later the same friend explained me that Perl was working much better on Linux and that Debian Linux installs it by default so I should try this one. I had no idea of what Linux was, but given how I loved Perl, I was eager to try his advice. So I got myself a Tri-Linux CD with Debian/RedHat/Slackware on it and started the installation process (which involved preparing boot floppies). But I did not manage to get the graphical interface working despite lots of fiddling with Xfree86 s configuration file. So I ended up installing RedHat and used it for a few months. But since many of the smart guys in my Usenet community were Debian users, I persisted and finally managed to get it to work! After a few months of usage, I was amazed at everything that was available for free and I wanted to give back. I filed my first bug report in July 1998, I created my first Debian packages in August 1998 and I got accepted as an official Debian developer in September 1998 (after a quick chat over the phone with Martin Schulze or James Troup I never understood the name of my interlocutor on the phone and I was so embarassed to have to use my rusty English over the phone that I never asked). That s right, it took me less than 3 months to become a Debian developer (I was 19 years old back then). I learned a lot during those months by reading and interacting with other Debian developers. Many of those went away from Debian in the mean time but some of them are still involved (Joey Hess, Manoj Srivastava, Ian Jackson, Martin Schulze, Steve McIntyre, Bdale Garbee, Adam Heath, John Goerzen, Marco D Itri, Phil Hands, Lars Wirzenius, Santiago Vila, Matthias Klose, Dan Jacobowitz, Michael Meskes, ). My initial Debian work was centered around Perl: I adopted dpkg-ftp (the FTP method for dselect) because it was written in Perl and had lots of outstanding bug reports. But I also got involved in more generic Quality Assurance work and tried to organize the nascent QA team. It was all really a lot of fun, I could take initiatives and it was clear to me that my work was appreciated. I don t know if you find this story interesting but I had some fun time digging through archives to find out the precise dates if you want to learn more about what I did over the following years, I maintain a webpage for this purpose.

One comment Liked this article? Click here. My blog is Flattr-enabled.

31 July 2012

Christian Perrier: Discovering a new package: HotelDruid

What is good in "my" job in Debian are opportunities to discover new interesting packages. While surveying the completion of debconf translations in unstable, I thus noticed a new package named "hoteldruid", that has a few questions and interaction with users. After my usual mumble because French and a few other languages were *finally* virtually 100% in unstable for a few days...... I went on my usual task in such cases: propose a review of debconf templates and package description to my fellow debian-l10n-english co-workers (/me bends to Justin B. Rye, our tireless, picky, efficient and very clever Master Reviewer for over 5 years now). Then I discovered what HotelDruid is about: this is a piece of PHP-based software meant to manage....an hotel or bed and breakfast...or any kind of such facility. Real end-user software. Really useful software. For real people...:-) Not the gazillionth obscure development language, or yet another encryption library, or yet another mysterious virtualization thing used by 10 people in the world (even if they host thousands of machines). These are the free software pieces I like the most. Probably Marco Maria Francesco De Santis (the upstream author and Debian package maintainer) somewhere in, I guess, Italy is running a small B&B (or maybe his parents, or his wife/cousin/whatever) *and* is a free software addict. And he wanted to run his business with free software. Same for this French genealogist who once wanted to display his data over the web (and make a real use of that obscure Ocaml language.....yes, pun itended to my friends, here). Of the person who wrote LedgerSMB to manage his business. Or those who use free software to manage hospitals (hi Andreas) or schools (hi DebianEdu folks...and special hi to Petter). Real software for real people. Of course, developed with obscure geeky things used only by those weird geeks who like to sometimes travel half a planet to just gather together and develop the best free operating system ever. Guess what? I like this! And guess what? I proposed the HotelDruid author to check whether we could imrove....translation, of course..:-)

6 July 2012

Marco Silva: C-d on UNIX

[...] read also says how many bytes of the file were returned, so end of file is assumed when a read says "zero bytes are being returned." [...] When a program reads from your terminal, each input line is given to the program by the kernel only when you type its newline (i.e, press RETURN). [...] [...] Now try something different: type some characters and then a *ctl*-d rather than a RETURN: $ cat -u
123
ctl-d123 cat prints the characters out immediately. *ctl*-d says, "immediately send the characters I have typed to the program that is reading from my terminal." The *ctl*-d itself is not sent to the program, unlike a newline. Now type a second ctl-d, with no other chracters: $ cat -u
123
ctl-d123ctl-d$ The shell responds with a prompt, because cat read no characters, decided that meant end of file, and stopped. ctl-d sends whatever you have typed to the program that is reading from the terminal. If you haven't typed anything, the program will therefore read no characters, and that looks like the end of the file. That is why *ctl*-d logs you out --- the shell sees no more input. Of course, *ctl*-d is usually used to signal an end-of-file but it is interesting that is has a more general function.
Brian W. Kernighan, Rob Pike. The UNIX Programming Environment. Prentice-Haskell Software Series, 1984, section 2.1, pages 44-45.

6 June 2012

Petter Reinholdtsen: Web service to look up HP and Dell computer hardware support status

A few days ago I reported how to get the support status out of Dell using an unofficial and undocumented SOAP API, which I since have found out was discovered by Daniel De Marco in february. Combined with my web scraping code for HP, Dell and IBM from 2009, I got inspired and wrote a web service based on Scraperwiki to make it easy to look up the support status and get a machine readable result back. This is what it look like at the moment when asking for the JSON output:
% GET https://views.scraperwiki.com/run/computer-hardware-support-status/?format=json&vendor=Dell&servicetag=2v1xwn1
supportstatus( "servicetag": "2v1xwn1", "warrantyend": "2013-11-24", "shipped": "2010-11-24", "scrapestamputc": "2012-06-06T20:26:56.965847", "scrapedurl": "http://143.166.84.118/services/assetservice.asmx?WSDL", "vendor": "Dell", "productid": "" )
%
It currently support Dell and HP, and I am hoping for help to add support for other vendors. The python source is available on Scraperwiki and I welcome help with adding more features.

29 February 2012

Marco d'Itri: The role of niche architectures and toy ports in Debian

In the last few days, people on the debian-devel mailing list have been debating again (and again, and again...) how much the ports and architectures used only by an handful of people should influence the direction of the project. With this post I want to briefly summarize my position. The reason we support niche architectures and toy ports is that that some people are interested in doing the work, and that these do not hinder development of the architectures that people actually use. When m68k started inconveniencing too much the real world development, the port was killed and the sky did not fall, notwithstanding the complaints of the few retrocomputing enthusiasts who still used and developed it. I do not mind if some people like to play with kFreeBSD and Hurd as long as they do not inconvenience me too much, both as a user and as a developer. While it is true that targeting multiple architectures helps finding bugs, after we have covered all the useful combinations of endianity, size of variables and char signedness the incremental benefits of adopting a new architecture just for its sake are minimal. I am a Red Hat customer (also because of Debian shortcomings), I do not like it much and I do not want to be one forever: please do not kill my hopes.

2 February 2012

Marco Silva: Intuitive python

>>> x = ['a', 'b']
>>> x
['a', 'b']
>>> x += 'c'
>>> x
['a', 'b', 'c']
>>> x += ''
>>> x
['a', 'b', 'c']
>>> x += ['']
>>> x
['a', 'b', 'c', '']

Marco Silva: Using Horizontal Sharding in SQLAlchemy to display multiple DBs

My problem was: I had a number of databases generated in different machines and I wanted to query them as if they were one, using the database the data came from as a field while querying and while showing results. The databases are SQLite3 files, generated using SQLAlchemy in a Python program. I solved this by using SQLAlchemy, which was good because I could use the same ORM mapping that the program used. I noticed that the Horizontal Sharding SQLAlchemy extension would fit well the problem, although not perfectly. I had to make some changes in some classes of this extension, and now it works fine. It was possible to filter the data using database as a criteria, but I couldn't get the database information from each line of a query result. I made a simple patch to SQLAlchemy, which wasn't likely to be introduced in the distribution, but worked for me, and sent it to its bug tracker. The change was included in SQLAlchemy in a very different fashion, as expected, but since I'm using the released version from SQLAlchemy, I kept on using my version of the patch. I don't want to do direct changes in SQLAlchemy source code, so I made the change in my program:
class ShardedSessionShardId(ShardedSession):
    def __init__(self, *args, **kwargs):
        super(ShardedSessionShardId, self).__init__(*args, **kwargs)
        self._query_cls = ShardedQueryShardId
class ShardedQueryShardId(ShardedQuery):
    def _execute_and_instances(self, context):
        if self._shard_id is not None:
            result = self.session.connection(
                            mapper=self._mapper_zero(),
                            shard_id=self._shard_id).execute(context.statement, self._params)
            news = list(self.instances(result, context))
            for new in news:
                new.shard_id = self._shard_id
            return iter(news)
        else:
            partial = []
            for shard_id in self.query_chooser(self):
                result = self.session.connection(
                            mapper=self._mapper_zero(),
                            shard_id=shard_id).execute(context.statement, self._params)
                news = list(self.instances(result, context))
                for new in news:
                    new.shard_id = shard_id
                partial = partial + news
            # if some kind of in memory 'sorting'
            # were done, this is where it would happen
            return iter(partial)
create_session = sessionmaker(class_=ShardedSessionShardId)
Another problem is that I had to make each result be included in the query, even if two results from different DBs have the same primary key. I achieved this by changing two classes: WeakInstanceDict, and Mapper. For using the new WeakInstanceDict, I had again to change the ShardedSession variation:
class WeakInstanceDictNoIdentity(WeakInstanceDict):
    def add(self, state):
        # if state.key in self:
        #     if dict.__getitem__(self, state.key) is not state:
        #         raise AssertionError("A conflicting state is already "
        #                             "present in the identity map for key %r"
        #                             % (state.key, ))
        # else:
            dict.__setitem__(self, state.key, state)
            self._manage_incoming_state(state)
class ShardedSessionShardId(ShardedSession):
    def __init__(self, *args, **kwargs):
        super(ShardedSessionShardId, self).__init__(*args, **kwargs)
        self._query_cls = ShardedQueryShardId
        self._identity_cls = WeakInstanceDictNoIdentity
        self.identity_map = self._identity_cls()
To start using the new Mapper, I simply replaced each call to mapper with MapperNoIdentity:
class MapperNoIdentity(Mapper):
    def _instance_processor(self, context, path, adapter,
                                polymorphic_from=None, extension=None,
                                only_load_props=None, refresh_state=None,
                                polymorphic_discriminator=None):
        """Produce a mapper level row processor callable
           which processes rows into mapped instances."""
        pk_cols = self.primary_key
        if polymorphic_from or refresh_state:
            polymorphic_on = None
        else:
            if polymorphic_discriminator is not None:
                polymorphic_on = polymorphic_discriminator
            else:
                polymorphic_on = self.polymorphic_on
            polymorphic_instances = util.PopulateDict(
                                        self._configure_subclass_mapper(
                                                context, path, adapter)
                                        )
        version_id_col = self.version_id_col
        if adapter:
            pk_cols = [adapter.columns[c] for c in pk_cols]
            if polymorphic_on is not None:
                polymorphic_on = adapter.columns[polymorphic_on]
            if version_id_col is not None:
                version_id_col = adapter.columns[version_id_col]
        identity_class = self._identity_class
        def identity_key(row):
            return identity_class, tuple([row[column] for column in pk_cols])
        new_populators = []
        existing_populators = []
        load_path = context.query._current_path + path
        def populate_state(state, dict_, row, isnew, only_load_props):
            if isnew:
                if context.propagate_options:
                    state.load_options = context.propagate_options
                if state.load_options:
                    state.load_path = load_path
            if not new_populators:
                new_populators[:], existing_populators[:] = \
                                    self._populators(context, path, row,
                                                        adapter)
            if isnew:
                populators = new_populators
            else:
                populators = existing_populators
            if only_load_props:
                populators = [p for p in populators
                                if p[0] in only_load_props]
            for key, populator in populators:
                populator(state, dict_, row)
        session_identity_map = context.session.identity_map
        if not extension:
            extension = self.extension
        translate_row = extension.get('translate_row', None)
        create_instance = extension.get('create_instance', None)
        populate_instance = extension.get('populate_instance', None)
        append_result = extension.get('append_result', None)
        populate_existing = context.populate_existing or self.always_refresh
        if self.allow_partial_pks:
            is_not_primary_key = _none_set.issuperset
        else:
            is_not_primary_key = _none_set.issubset
        def _instance(row, result):
            if translate_row:
                ret = translate_row(self, context, row)
                if ret is not EXT_CONTINUE:
                    row = ret
            if polymorphic_on is not None:
                discriminator = row[polymorphic_on]
                if discriminator is not None:
                    _instance = polymorphic_instances[discriminator]
                    if _instance:
                        return _instance(row, result)
            # determine identity key
            if refresh_state:
                identitykey = refresh_state.key
                if identitykey is None:
                    # super-rare condition; a refresh is being called
                    # on a non-instance-key instance; this is meant to only
                    # occur within a flush()
                    identitykey = self._identity_key_from_state(refresh_state)
            else:
                identitykey = identity_key(row)
            # instance = session_identity_map.get(identitykey)
            # if instance is not None:
            #     state = attributes.instance_state(instance)
            #     dict_ = attributes.instance_dict(instance)
            #     isnew = state.runid != context.runid
            #     currentload = not isnew
            #     loaded_instance = False
            #     if not currentload and \
            #             version_id_col is not None and \
            #             context.version_check and \
            #             self._get_state_attr_by_column(
            #                     state,
            #                     dict_,
            #                     self.version_id_col) != \
            #                             row[version_id_col]:
            #         raise orm_exc.ConcurrentModificationError(
            #                 "Instance '%s' version of %s does not match %s"
            #                 % (state_str(state),
            #                     self._get_state_attr_by_column(
            #                                 state, dict_,
            #                                 self.version_id_col),
            #                         row[version_id_col]))
            # elif refresh_state:
            if refresh_state:
                # out of band refresh_state detected (i.e. its not in the
                # session.identity_map) honor it anyway.  this can happen
                # if a _get() occurs within save_obj(), such as
                # when eager_defaults is True.
                state = refresh_state
                instance = state.obj()
                dict_ = attributes.instance_dict(instance)
                isnew = state.runid != context.runid
                currentload = True
                loaded_instance = False
            else:
                # check for non-NULL values in the primary key columns,
                # else no entity is returned for the row
                if is_not_primary_key(identitykey[1]):
                    return None
                isnew = True
                currentload = True
                loaded_instance = True
                if create_instance:
                    instance = create_instance(self,
                                                context,
                                                row, self.class_)
                    if instance is EXT_CONTINUE:
                        instance = self.class_manager.new_instance()
                    else:
                        manager = attributes.manager_of_class(
                                                instance.__class__)
                        # TODO: if manager is None, raise a friendly error
                        # about returning instances of unmapped types
                        manager.setup_instance(instance)
                else:
                    instance = self.class_manager.new_instance()
                dict_ = attributes.instance_dict(instance)
                state = attributes.instance_state(instance)
                state.key = identitykey
                # manually adding instance to session.  for a complete add,
                # session._finalize_loaded() must be called.
                state.session_id = context.session.hash_key
                session_identity_map.add(state)
            if currentload or populate_existing:
                if isnew:
                    state.runid = context.runid
                    context.progress[state] = dict_
                if not populate_instance or \
                        populate_instance(self, context, row, instance,
                            only_load_props=only_load_props,
                            instancekey=identitykey, isnew=isnew) is \
                            EXT_CONTINUE:
                    populate_state(state, dict_, row, isnew, only_load_props)
            else:
                # populate attributes on non-loading instances which have
                # been expired
                # TODO: apply eager loads to un-lazy loaded collections ?
                if state in context.partials or state.unloaded:
                    if state in context.partials:
                        isnew = False
                        (d_, attrs) = context.partials[state]
                    else:
                        isnew = True
                        attrs = state.unloaded
                        # allow query.instances to commit the subset of attrs
                        context.partials[state] = (dict_, attrs)
                    if not populate_instance or \
                            populate_instance(self, context, row, instance,
                                only_load_props=attrs,
                                instancekey=identitykey, isnew=isnew) is \
                                EXT_CONTINUE:
                        populate_state(state, dict_, row, isnew, attrs)
            if loaded_instance:
                state._run_on_load(instance)
            if result is not None and \
                        (not append_result or
                            append_result(self, context, row, instance,
                                    result, instancekey=identitykey,
                                    isnew=isnew)
                                    is EXT_CONTINUE):
                result.append(instance)
            return instance
        return _instance
I had to include some auxiliary definitions to make the rewrites work:
_none_set = frozenset([None])
_runid = 1L
_id_lock = util.threading.Lock()
def _new_runid():
    global _runid
    _id_lock.acquire()
    try:
        _runid += 1
        return _runid
    finally:
        _id_lock.release()
It would be good to be able to set these identity requirements as a parameter. My last problem was selecting more than one database to search. setshard only worked for one, so I created a new field in query, called shards, and checked for it on querychooser:
def query_chooser(query):
    try:
        return query.shards
    except AttributeError:
        pass
    return tcs.keys()
So, when I want to look only in a list of shards, I set this field. I'm aware that this is not a recommended python idiom, but, well, it works fine.

Marco Silva: Problems with popup-menu signal in GTK+

I'm using GTK+ in my paid work, which is being cool because I like this library. It's not the first time I use it for paid work, and I like these opportunities to learn more about it and possibly report bugs. It's not easy to find bugs in such a widely used library, so much of them end up being simple cosmetic mistakes, like this one. Some others are only a question of interpretation, like this other, which even when I submitted a patch, it was not applied. Yesterday I couldn't make popup-menu create a, well, popup menu. It was working with button-press-event, but not with popup-menu. I created a small code sample to illustrate this problem and asked about it in #pygtk@irc.gimp.org --- yes, I'm using Python:
import gtk
import gobject
def menu():
  item = gtk.ImageMenuItem('gtk-add')
  item.show()
  menu = gtk.Menu()
  menu.add(item)
  return menu
def on_view_popup_menu(obj, *data):
  print 'popup-menu'
  menu().popup(None, None, None, 0, 0)
def on_view_button_press_event(obj, *data):
  print 'button-press-event'
  menu().popup(None, None, None, 0, 0)
store = gtk.ListStore(gobject.TYPE_STRING)
store.append(('teste',))
view = gtk.TreeView(store)
view.append_column(gtk.TreeViewColumn('Coluna', gtk.CellRendererText(), text=0))
view.connect('button-press-event', on_view_button_press_event)
view.connect('popup-menu', on_view_popup_menu)
view.show()
window = gtk.Window()
window.add(view)
window.show()
gtk.main()
When I clicked in the window, with any button in this case, 'button-press-event' is printed to stdout and the menu is displayed. When I press the Menu key, or Shift+F10, 'popup-menu' is printed to stdout, but the menu is not displayed. After getting no response in #pygtk, I decided to try #gtk+. Before that, I thought about trying the same problem in C to see if it was not a problem in the bindings, or in how I was using it. The code:
#include <gtk/gtk.h>
GtkMenu *menu()  
  GtkImageMenuItem *image_menu_item =
      GTK_IMAGE_MENU_ITEM(
          gtk_image_menu_item_new_from_stock(GTK_STOCK_ADD, NULL));
  gtk_widget_show(GTK_WIDGET(image_menu_item));
  GtkMenu *menu = GTK_MENU(gtk_menu_new());
  gtk_container_add(GTK_CONTAINER(menu), GTK_WIDGET(image_menu_item));
  return menu;
 
void on_tree_view_popup_menu(GtkWidget *widget, gpointer user_data)  
  printf("popup-menu\n");
  gtk_menu_popup(
      menu(), NULL, NULL, NULL, NULL, 0, gdk_event_get_time(NULL));
 
void
on_tree_view_button_press_event(
  GtkWidget *widget, GdkEventButton *event, gpointer user_data)  
  printf("button-press-event\n");
  gtk_menu_popup(
      menu(), NULL, NULL, NULL, NULL, 0, gdk_event_get_time(NULL));
 
int main(int argc, char **argv)  
  gtk_init(&argc, &argv);
  GtkListStore *list_store = gtk_list_store_new(1, G_TYPE_STRING);
  GtkTreeIter iter;
  gtk_list_store_append(list_store, &iter);
  gtk_list_store_set(list_store, &iter, 0, "teste", -1);
  GtkTreeView *tree_view =
      GTK_TREE_VIEW(
          gtk_tree_view_new_with_model(GTK_TREE_MODEL(list_store)));
  gtk_tree_view_append_column(
      tree_view,
      gtk_tree_view_column_new_with_attributes(
          "Coluna", gtk_cell_renderer_text_new(), "text", 0, NULL));
  g_signal_connect(
      tree_view, "popup-menu", G_CALLBACK(on_tree_view_popup_menu), NULL);
  g_signal_connect(
      tree_view,
      "button-press-event",
      G_CALLBACK(on_tree_view_button_press_event),
      NULL);
  gtk_widget_show(GTK_WIDGET(tree_view));
  GtkWindow *window = GTK_WINDOW(gtk_window_new(GTK_WINDOW_TOPLEVEL));
  gtk_container_add(GTK_CONTAINER(window), GTK_WIDGET(tree_view));
  gtk_widget_show(GTK_WIDGET(window));
  gtk_main();
 
Wow. The difference between the codes is impressive. I'm not that used to code in GTK+ using C, and I may have missed something here, but the C code is so verbose that I get tired only by looking at it. I'm glad we have other languages, and GTK+ bindings for them. As expected, the behavior was the same, and I asked in #gtk+ and got no answer. Yes, I did read the documentation before asking, and no, I didn't read it carefully enough. Maybe because of coming from Python, I didn't checked very much the type of the callback function for the signals, and didn't notice the point about the return value: "Returns: TRUE if a menu was activated". And that was not where I was looking for the problem, since I thought the return value was usually related to the treatment of the signal by another functions. Anyway, I changed the type of on_tree_view_popup_menu to return gboolean and returned TRUE. It works! I also updated the type of on_tree_view_button_press_event to do the same, just for compliance. In Python, a simple return True would do it. I know I could use the event parameter in on_tree_view_button_press_event, but the point was exactly to run the same code and see if it would work in one case and not in the other, as happened.

Next.

Previous.